Foldsformer: Learning Sequential Multi-Step Cloth Manipulation With Space-Time Attention

Sequential multi-step cloth manipulation is a challenging problem in robotic manipulation, requiring robot to perceive the state and plan sequence of chained actions leading desired state. Most previous works address this goal-conditioned way, goal observation must be given for each specific task configuration, which not practical efficient. Thus, we present novel planning framework named Foldformer. Foldformer can complete similar tasks with only general demonstration utilize space-time attention mechanism capture instruction information behind demonstration. We experimentally evaluate Foldsformer on four representative sequential show that significantly outperforms state-of-the-art approaches simulation. even when configurations (e.g., size pose) vary from demonstrations. Furthermore, our approach transferred simulation real world without additional training or domain randomization. Despite rectangular clothes, also generalize unseen shapes (T-shirts shorts).